Remarks 



This Preliminary Amendment cancels without prejudice original claims 1 to 1 1 in the 
underlying PCT Application No. PCT/DE03/01 111. This Preliminary Amendment adds new 
claims 12-23. The new claims, inter alia, conform the claims to United States Patent and 
Trademark Office rules and do not add new matter to the application 

In accordance with 37 C.F.R. § 1.125(b), the Substitute Specification (including the 
Abstract, but without the claims) contains no new matter. The amendments reflected in the 
Substitute Specification (including Abstract) are to conform the Specification and Abstract to 
U.S. Patent and Trademark Office rules or to correct informalities. As required by 37 C.F.R. 
§ 1.121(b)(3)(h) and § 1.125(c), a Marked Up Version Of The Substitute Specification 
comparing the Specification of record and the Substitute Specification also accompanies this 
Preliminary Amendment. Approval and entry of the Substitute Specification (including 
Abstract) are respectfully requested. 

The underlying PCT Application No. PCT/DE03/01 1 1 1 includes an International 
Search Report, dated August 8, 2003, a copy of which is included. The Search Report 
includes a list of documents that were considered by the Examiner in the underlying PCT 
application. 

Applicants assert that the subject matter of the present application is new, 
non-obvious, and useful. Prompt consideration and allowance of the application are 
respectfully requested. 



Respectfully Submitted, 
KENYON & KENYON 
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METHOD AND DEVICE FOR CALIBRATING SENSORS IN A 

MOTOR VEHICLE 



Field Of The Invention 

The present invention relates to vehicle sensors, and in 
particular relates to a method and device for calibrating 
sensor in a motor vehicle. 

Background Information Background Information 
The use of a plurality of sensors in motor vehicles for 
detecting the vehicle environment is known . Preferably, — fefee 
uac of image acnoor ayotcmo, — infrared ocnooro, — ultraoound 
ocnooro and radar ocnaora ia planned. For example, radar 
sensors are used to detect the vehicle environment in the 
direction of travel. This makes it possible to regulate the 
speed of the vehicle, the road speed being adapted to slower 
moving vehicles when the radar sensor detects them in the 
heading range of the motor vehicle. 

For the sensors to function properly, they must be aligned 
with respect to the motor vehicle and the vehicle environment. 
To this end, the sensors are calibrated in an environment 
similar to a laboratory after they have been installed in the 
vehicle. For example, a method for aligning the beam pattern 
of a radar sensor is known from described in German Patent BE 
Application No. 197 07 590 G3-. A calibration object having the 
reference features for determining the calibration data is 
used for calibration. References to the calibration of 
additional sensors are lacking here. — not mentioned . 
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Advantagco of the — Invention Summary Of The Invention 
The method deacribed below of the present invention has the 
advantage that through the use of a common calibration object 
for various sensor types, the sensors are aligned on a 
coordinate system. This makes it possible for the data of the 
sensors calibrated in this manner to be used jointly without 
discrepancies in the measurements. 

This is advantageous in particular in the use of a radar 

sensor and an image sensor system. For example, it is possible 
to combine the data detected by a radar sensor concerning a 
vehicle traveling in front of the vehicle with the data o# 
from the image sensor system-7 — which hao — identified concerning 
the same vehicle as an image object. It is possible to use 
this for a redundant, error reducing corrective analysis of 
the data of both sensors. 

Advantageouoly , — the deacribed The method and the device 
according to the present invention reduce the time and effort 
required for calibrating sensors that are situated in a motor 
vehicle, for example. Preferably, — art It is possible to perform 
the calibration in one operation during production or when a 
motor vehicle visits a workshop after the installation of a 
radar sensor and an image sensor system. The use of a single 
calibration object may advantageously make it possible to 
determine the calibration data of both sensors simultaneously, 
i.e., the calibration data is determined in one operation in a 
close temporal relationship. This results in a reduction of 
the time required to calibrate sensors during the production 
or repair of a motor vehicle. This is associated with a 
reduction in the production or repair costs. 

In an advantagcoua manner, — fcfee The calibration data 4r& 
prcf crably may be used further by downstream systems. The 
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calibration data may be stored and/or analyzed and/or 
displayed and/or transmitted and/or further processed. 
Furthermore, the sensors themselves arc able to can use the 
calibration data further in an advantageous manner. For 
example, it is possible in a radar sensor to use the 
calibration data to correct the discrepancy of the sensor axis 
in relation to the direction of the vehicle longitudinal axis. 
In an image sensor system, it is possible to use the 
calibration data to correct the image data automatically. The 
extrinsic parameters, preferably such as the geometrical 
installation position and/or and the direction of the sensor 
axis, and/or intrinsic parameters, preferably such as the 
camera's principal point and/or ^ the camera constant and/or and 
the distortion, are determined here as calibration data. 

In an advantageous manner in particular, it is possible to use 
at least one additional sensor as a reference for calibrating 
a sensor. The calibration data of the sensor is formed from 
the data of the detected calibration object and additionally 
from the measured data of at least one additional sensor, this 
reference sensor using the same calibration object. In a motor 
vehicle having a radar sensor and an image sensor system, it 
is possible, for example, to use the data of the image sensor 
system as a reference for determining the calibration data of 
the radar sensor. In an advantageous manner in particular, it 
is possible, for example, during a repair of the radar sensor, 
with a recalibration being necessary, to form the calibration 
data of the radar sensor from the measured data of the image 
sensor system and the data of the calibration object detected 
by the radar sensor. This advantageously results in an 
increase in the measuring accuracy of the radar sensor. 

Of particular advantage io the It may be particularly 
advantageous to use of- a single calibration object— which has 



NY01 730682 vl 



3 



MARKED -UP VERSION OF THE 
SUBSTITUTE SPECIFICATION 



at least one reference feature for each sensor for the at 
least two sensors, the at least two sensors being of a 
different sensor type. The combination of the reference 
features into one calibration object for calibrating at least 
two sensors of a different sensor type substantially 
facilitates the calibration, for example, in the production of 
a motor vehicle because it is now only necessary to align the 
motor vehicle to one calibration object. 

In addition, the combination of the reference features into 
one calibration object has the advantage that the calibration 
object may be designed in such a way that the physical 
position of the reference features in relation to each other 
is known. In an advantageous manner, this forms the basis for 
using the measured data of a sensor as a reference for another 
sensor . 

In an advantagcouo manner, — a A triple mirror 4r& may be used as 
a reference feature for the radar sensor. Triple mirrors are 
distinguished by three reflection surfaces, which are 
perpendicular in relation to each other, which make a parallel 
reflection of the incident radar beam possible, the radar beam 
being reflected in a wide angular range parallel to the 
incident beam with reference to the reflection surfaces. 

In one variation of the calibration object, calibration marks 
are placed on the triple mirror. This advantageously makes it 
possible to use the triple mirror as a reference feature for 
the radar sensor and the image sensor system. The use of this 
device makes any additional reference features for the image 
sensor system unnecessary. Furthermore, this system makes the 
physical position of the reference features known for the 
radar sensor and the image sensor system. It is thus possible 
to use the radar sensor as a reference for the image sensor 
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system and the image sensor system as a reference for the 
radar sensor . 



Additional advantages arc derived from the — following 
dcocription of exemplary cmbodimento with reference to the 
figurco , — and from the dependent claima . 

Brief Description of The Drawings 

Drawing 

The prcocnt invention will be explained in greater detail 
below with reference to the cmbodimento ohown in the drawing 
in which : 

Figure 1 shows a general drawing an embodiment of the a device 
for calibrating sensors in the motor vehicle according to the 
present invention with reference to which the method io aloo 
described, ^ 

Figure 2 shows a flowchart of an embodiment of the method for 
calibrating sensors according to the present invention. 

Figure 3 shows a block diagram of the device for calibrating 
sensors according to the present invention . 

Detailed Description Dcocription of Exemplary Embodimcnto 

In a general drawing, Figure 1 shows a preferred an exemplary 
embodiment for calibrating sensors 14, 16. On and/or in a 
motor Motor vehicle 12 arc located includes a radar sensor 14, 
which io preferably may be attached to the bumper, and an 
image sensor system 16, which io preferably may be situated in 
the vehicle interior. Both sensors 14, 16 have a detection 
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range 15, 17, each of which is aligned in the direction of 
vehicle longitudinal axis 11 in the direction of travel of 
motor vehicle 12. In the preferred an exemplary embodiment, 
the direction of detection ranges 15, 17 is determined by the 
sensor axes. In radar sensor 14, the direction of maximum 
sensitivity of radar sensor 14 determines the sensor axis 
while in image sensor system 16, the optical axis is the 
sensor axis. In the preferred exemplary embodiment, detection 
range 15 of radar sensor 14 coincides with detection range 17 
of image sensor system 16. In the preferred exemplary 
embodiment , — image Image sensor system 16 includes may include 
one image sensor whose design is based, for example, on CMOS 
or CCD technology. Motor vehicle 12 is aligned in front of a 
calibration object 10 in such a way that radar sensor 14 and 
image sensor system 16 detect reference features 18, 20 on 
calibration object 10. In the preferred exemplary embodiment, 
it io It may be sufficient that radar sensor 14 detects 
reference feature 18 and image sensor system 16 detects 
reference feature 20. Calibration object 10 is a flat or 
three-dimensional system of reference features 18, 20, which 
are connected by a mechanical mounting device. In the 
preferred exemplary embodiment, — both Both reference features 
18, 20 a-re may be mounted on a flat wall, which is used as a 
mounting device. Reference features 18, 20 are designed in 
such a way that they are always detectable by radar sensor 14 
and image sensor system 16. Reference feature 18, which is 
detected by radar sensor 14, has characteristic properties, to 
the preferred exemplary embodiment, — radar ocnoor 14 — ie Radar 
sensor 14 may be a radar receiver— which is combined with a 
transmitter for radar radiation. Reference feature 18 must be 
is designed in such a way that it reflects the incident radar 
beam back to radar sensor 14- — In the preferred exemplary 
embodiment , — it — ar s, and may be implemented as a triple mirror. 
Reference feature 20, which is detected by image sensor system 
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16 , also has particular properties. Preferably , — reference 
Reference feature 2 0 haa may have a high contrast and/or a 
reflective surface and/or it hao a particular shape, such as a 
circular or rectangular shape, for example. In order to 
calibrate image sensor system 16, one to any desired number of 
reference features 2 0 is ncccooary, — preferably employed, such 
as between ten and 50 reference features 20 being uocd. — In the 
preferred exemplary embodiment, — fo r . For example, 4 0 reference 
features 20 arc may be used to determine the calibration data 
of image sensor system 16, their physical location being 
known. This makes a redundant determination of the calibration 
data for image sensor system 16 possible. 

Figure 2 shows a: an exempl ary flowchart of the method for 
calibrating sensors 14, 16 according to the present invention . 
In block 22, sensors 14, 16 are aligned in relation to 
calibration object 10. In the preferred exemplary embodiment, 
thio ia This may be done by aligning motor vehicle 12, on 
which radar sensor 14 and image sensor system 16 arc is 
located, in relation to calibration object 10. Alternatively, 
it is possible to align calibration object 10 in relation to 
motor vehicle 12. In block 24, reference features 18, 20 of 
calibration object 10 are detected by sensors 14, 16. Block 26 
is used to determine the calibration data for radar sensor 14 
and image sensor system 16 from the data determined of 
detected calibration object 10. The calibration data are 
determined simultaneously, i.e., the calibration data are 
determined in one operation in a close temporal relationship. 
Prcf crably , — the The direction of the sensor axis in relation 
to vehicle longitudinal axis 11 and/or the intensity of the 
radar beam arc may be determined as calibration data in radar 
sensor 14 . 
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To determine the calibration data, known reference feature 20 
is recorded as an image in block 24 in image sensor system 16 
and the individual parameters of the calibration data are 
calculated in a computation of adjustment in subsequent block 
26. In the preferred an exemplary embodiment, extrinsic 
parameters, preferably such as the geometric installation 
position and/or^ the direction of the sensor axis, and/or 
intrinsic parameters, preferably such as the camera's principal 
point and/or ^ the camera constant and/or and the distortion, 
are determined as calibration data in a single image sensor 
system 16. The determined calibration data is further 
processed in block 28. The calibration data may preferably be 
stored and/ or_ ^ analyzed and/or^ displayed and/or^ transmitted 
and/or further processed by at least one downstream system 34. 
In addition, the calibration data is further used directly for 
calibration in participating sensors 14, 16. In doing so, it 
is possible to process and/ or and store the calibration data. 
In the preferred exemplary embodiment, The image sensor system 
16 uoeo may use the calibration data for the automatic 
correction of the images in the operation of image sensor 
system 16. In the preferred exemplary embodiment, radar sensor 
14 uses the calibration data for the automatic correction of 
the deviation of the sensor axis in relation to the direction 
of vehicle longitudinal axis 11. As an alternative, it is 
possible to display the determined calibration data and thus, 
for example, to make it possible for an automotive mechanic to 
use the calibration data for the mechanical adjustment of 
radar sensor 14. This procedure is conceivable possible during 
the manufacture or the repair of a motor vehicle 12 . 

Figure 3 shows a block diagram of an exemplary embodiment of 
the device for calibrating sensors 14, 16 according to the 
present invention . In the preferred exemplary embodiment, a 
radar sensor 14 and an image sensor system 16 detect a 
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calibration object 10. The calibration data for the two 
sensors 14, 16 is determined in two analyzing units 30, 32. 
The calibration data is further utilized in a downstream 
system 34 . 

The method described above and the device for calibrating 
sensors 14, 16 are not limited to radar sensors 14 and image 
sensor systems 16 in motor vehicles 12. Instead, sensors 14, 
16 may be calibrated in systems outside of motor vehicle 12, 
sensors 14, 16 being characterized by a three-dimensional 
detection range 15, 17 and preferably having one sensor axis. 
In particular, the described method and the device may be used 
in radar sensors 14 and/ or ^ image sensor systems 16 and/ or ± 
light sensors and/or ± ultrasound sensors and/or LIDAR sensors. 
At the same time the method may also be implemented with more 
than two sensors 14, 16. With more than two sensors 14, 16, 
sensors 14, 16 of the same sensor type using the same 
reference features 18, 2 0 may also be used. Two possible 
systems are, for example, at least one radar sensor 14 
including at least two image sensor systems 16 or at least two 
radar sensors 14 including at least one image sensor system 
16, sensors 14, 16 preferably being situated in a motor 
vehicle 12. If at least two image sensor systems 16 are used, 
additional calibration data is determined, which contain the 
physical position and/or the alignment of the sensor axes of 
image sensor systems 16 in relation to each other. 

The alignment of detection ranges 15, 17 of sensors 14, 16 is 
discretionary. Ao in the preferred exemplary embodiment, 
detection Detection ranges 15, 17 may include the vehicle 
environment. It is also conceivable possible to align 
detection ranges 15, 17 into the vehicle interior. Any desired 
mutual alignment of detection ranges 15, 17 of sensors 14, 16 
is possible. In particular, the identical or diametrically 
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opposed alignment of detection ranges 15, 17 is conceivable 
possible . Sensors 14, 16 may be situated both within motor 
vehicle 12 as well as outside of it. For example, it is 
possible to mount sensors 14, 16 in the interior of the 
vehicle and/or on the bumper and/or in the luggage compartment 
and/or in the engine compartment . The only prcrcquioitc It is 
useful in this context for uoing the device and the method ia 
that calibration object 10 muot be adjusted to the 
installation position and the alignment of detection ranges 
15, 17 of sensors 14, 16 by the physical positioning of 
reference features 18, 20. 

A coincidence of detection ranges 15, 17 of sensors 14, 16 is 
not a necessary prerequisite for implementing the method. In 
the case of detection ranges 15, 17 of sensors 14, 16 that do 
not coincide, calibration object 10 must simply be designed in 
such a way that a detection of necessary reference features 
18, 20 is possible in one operation for each individual sensor 
14, 16. It is not necessary for each sensor 14, 16 to detect 
entire calibration object 10 including all reference features 
18, 20. 

The method is also possible with moving sensors 14, 16 and/or 
with moving calibration object 10. For example, in the 
preferred exemplary embodiment, the device is also usable in a 
moving motor vehicle 12 . 

In one variation of the described method, a sensor 14, 16 is 
used as a reference for another sensor 14, 16. For example, 
the calibration data of radar sensor 14 may be formed as a 
reference from the measured data of already calibrated image 
sensor system 16 and the data of calibration object 10 
detected by radar sensor 14. The physical position of 
reference feature 18, which is ncccooary used to determine the 
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calibration data of radar sensor 14, may be determined by- 
image sensor system 16. The physical position of reference 
feature 18 is determined indirectly by analyzing reference 
feature 20 using the known physical positioning of reference 
features 18, 20 in relation to each other. 

In another variation of the described method, radar sensor 14 
is used as a reference for image sensor system 16. Already 
calibrated Calibrated radar sensor 14 is used for the range 
finding. Radar sensor 14 determines the distance of reference 
feature 18 from radar sensor 14. Image sensor system 16 uses 
the known physical positioning of reference features 18, 20 
and the determined distance of reference feature 18 from radar 
sensor 14 as a reference for determining the calibration data 
by analyzing calibration object 10. 

Ao in the preferred exemplary embodiment, — the The calibration 
data of sensors 14, 16 may be determined by separate analyzing 
units 30, 32. As an alternative, it is possible for a single 
analyzing unit to determine the calibration data. In another 
variation, the integration of analyzing units 30, 32 with 
sensors 14, 16 is conceivable possible . 

In one variation of the described method, calibration object 
10 may be connected directly to motor vehicle 12 by a 
mechanical link. This establishes the physical position of 
calibration object 10 in relation to motor vehicle 12. 

In another variation of the described method, the use of a 
triple mirror having calibration marks makes it possible to 
determine the calibration data of radar sensor 14 and of image 
sensor system 16 directly on the triple mirror because the 
triple mirror having calibration marks has both a reference 
feature 18 for radar sensor 14 and a reference feature 20 for 
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image sensor system 16. This makes it possible to use radar 
sensor 14 as a reference for image sensor system 16 and image 
sensor system 16 for radar sensor 14 for determining the 
calibration data. Moreover, additional reference features 20 
may be present for image sensor system 16. 

The number of reference features 18, 20 for individual sensors 
14, 16 establishes the number of determinable parameters of 
the calibration data. With more reference features 18, 20 for 
individual sensors 14, 16, more parameters are determinable, 
and/or an overdetermination and accordingly an increase in the 
precision for determining the parameters arc is also possible. 

The calibration by a single calibration object 10 in which the 
physical position of reference features 18, 20 is known, makes 
common use of sensors 14, 16 possible because the calibration 
data refers to a common coordinate system. In particular, when 
a radar sensor 14 and an image sensor system 16 are used in a 
motor vehicle 12 aa in the preferred exemplary embodiment , the 
measured data of radar sensor 14 may be combined with the data 
of image sensor system 16. In particular, vehicles traveling 
in front may be detected by both sensors 14, 16 and the 
measured data may be used for redundant error- reducing 
analyses . 
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Abotract 



Abstract Of The Disclosure 
A method and a device are described for calibrating at least 
two sensors (14 , — 3r6-K the sensors (14 , — tS-}- being of a different 
sensor type. Preferably these The sensors (14 , — 3r6-) — a^?e may be 
located in or on a motor vehicle (12) . It is possible to 
calibrate the at least two sensors (14 , — 3r6-)- in one operation 
by using a calibration object (10) containing reference 
features (18 , — 2-G->- for determining calibration data for the at 
least two sensors (14 , — 

(Figure 1) 
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